WHAT IS CLAIMED IS: 



1 . A computer-readable medium having computer executable 
instructions for automatically estimating a motion field for image frames in an 

5 image sequence, said computer executable instructions comprising: 

evaluating a first set of zero valued motion vector (MVs) for blocks in an 
image frame using background detection and determining a reliability of each 
MV; 

evaluating a second set of one or more candidate MVs for each block in 
10 the image frame for which the first set of zero valued MVs was deemed not 
reliable, said second set of MVs being determined using any of spatial and 
temporal neighbors of each of those blocks, and determining an optimal MV for 
each block of the second set and a reliability of each optimal MV; 

evaluating a third set of candidate MVs for all blocks in the image frame 
15 having MVs that were deemed not reliable using the first or the second set of 
MVs, said third set of MVs being determined using a block-based pattern search, 
and determining an optimal MV for each block of the third set; and 

outputting an optimal MV for each block using the reliable MVs from the 
first, second and third sets of MVs to form a motion field for the image frame. 

20 

2. The computer-readable medium of claim 1 wherein reliability of the 
zero valued MVs is determined by computing error values for the MVs for each 
block in the image frame, and comparing the computed error values to a first 

25 error threshold. 

3. The computer-readable medium of claim 2 wherein each block 
having a computed error value less than the first error threshold is deemed to 
have a reliable zero-valued MV. 

30 
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4. The computer-readable medium of claim 1 wherein each optimal 
MV for the second set is determined by computing error values for each 
candidate MV of each block, and selecting a candidate MV having a smallest 
computed error value; 

5 

5. The computer-readable medium of claim 1 wherein the reliability of 
the optimal MVs for the second set is determined by comparing the error value of 
each optimal MV with a second error threshold, wherein any optimal MV having a 
computed error value less than the second error threshold is deemed to be a 

10 reliable MV. 

6. The computer-readable medium of claim 1 wherein a second error 
threshold is computed as a minimum error value of the spatial and temporal 
neighbor blocks. 

15 

7. The computer-readable medium of claim 1 wherein the error value 
of the optimal MV for the second set is compared against a third threshold value, 
wherein: 

if the error value is larger than the third threshold value, then the 
20 third set of MVs comprises the entire search range of the block-based pattern 
search; and 

if the error value is smaller than the third threshold value, then the 
third set of MVs comprises a search range consisting of only immediate neighbor 
MVs of the optimal MV of the second set. 

25 

8. The computer-readable medium of claim 7 wherein the third threshold 
value is computed as a maximum of the computed error values of the spatial and 
temporal neighbor blocks. 
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9. The computer-readable medium of claim 1 wherein the pattern 
search is any of: 

a square MV search; 
a spiral MV search; 
5 a 2D logarithmic MV search, 

a diamond MV search; 
a hexagonal MV search; and 
a two-level hexagonal MV search. 

10 1 0. The computer-readable medium of claim 2 or claim 4 or claim 9 

further comprising an array wherein computed error values for each MV are 
stored as they are computed. 

1 1 . The computer-readable medium of claim 10 wherein the array is 
15 checked before computing an error value for any candidate MV to determine 
whether an error value for that candidate MV has already been computed, and 
wherein if the error value for that candidate MV has already been computed, then 
it is read back from the array instead of being computed. 

20 1 2. A system for estimating motion vectors (MVs) for blocks in an 

image frame, comprising: 

inputting an image sequence comprising two or more images to a first MV 
estimation stage; 

said first MV estimation stage using background detection to identify 
25 candidate MVs for background blocks in a current image frame from the image 
sequence; 

determining whether each of the candidate MVs for the background blocks 
are reliable; 

passing all blocks in the current image frame that are not background 
30 blocks, and all blocks in the current image frame that do not have reliable 
candidate MVs to a second MV estimation stage; 
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computing candidate MVs and an estimated error for all blocks in the 
second stage, and identifying the candidate MV having the lowest estimated 
error for each block in the second stage as a reliable MV for each particular block 
in the second stage; and 

outputting the reliable MVs from the first stage and the reliable MVs from 
the second stage as the estimated MVs for the current image frame. 

13. The system of claim 12 wherein the second MV estimation stage 
uses any of spatial and temporal neighbors of each block to compute the 
candidate MVs for each block in the second stage. 

14. The system of claim 12 wherein a third MV estimation stage uses a 
block-based pattern search to compute the candidate MVs for each block in the 
third stage for blocks having MVs determined to be unreliable in the second 
stage. 

15. The system of claim 12 wherein the second MV estimation stage 
uses a block-based pattern search to compute the candidate MVs for each block 
in the second stage. 

16. The system of claim 14 or claim 15 wherein the block-based pattern 
search is any of: 

a square MV search; 

a spiral MV search; 

a 2D logarithmic MV search. 

a diamond MV search; 

a hexagonal MV search; and 

a two-level hexagonal MV search. 
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17. A computer-implemented process for estimating motion vectors 
(MVs) for blocks comprising image frames in a sequence of image frames, 
comprising using a computer to: 

input an image sequence comprising two or more images to a first MV 
5 estimation stage; 

said first MV estimation stage using background detection to identify 
candidate MVs for background blocks in a current image frame from the image 
sequence; 

determine whether each of the candidate MVs for the background blocks 
10 are reliable; 

pass all blocks in the current image frame that are not background blocks, 
and all blocks in the current image frame do not have reliable candidate MVs to a 
second MV estimation stage; 

said second MV estimation stage using any of spatial and temporal 
15 neighbors of each block to compute candidate MVs for each block in the second 
stage; 

compute an estimated error for each candidate MV computed for the 
second stage and identifying the candidate MV having the lowest estimated error 
for each block in the second stage as a best candidate MV for each block; 
20 determine whether each of the best candidate MVs for each of the blocks 

in the second stage are reliable; 

pass all blocks in the current image frame from the second stage that do 
not have reliable candidate MVs to a third MV estimation stage; 

said third MV estimation stage using a block-based pattern search to 
25 compute candidate MVs for each block in the third stage; 

compute an estimated error for each candidate MV computed for the third 
stage and identifying the candidate MV having the lowest estimated error for 
each block in the third stage as a reliable MV for each block; and 

output the reliable MVs from the first, second and third stages as the 
30 estimated MVs for the current image frame. 

-36- 



18. The computer-implemented process of claim 17 wherein the block- 
based pattern search is any of: 
a square MV search; 
a spiral MV search; 
5 a 2D logarithmic MV search, 

a diamond MV search; 
a hexagonal MV search; and 
a two-level hexagonal MV search. 

10 1 9. The computer-implemented process of claim 1 7 further comprising 

an array, equal in size to the number of MVs to be searched, wherein estimated 
errors for each MV are stored as they are computed. 

20. The computer-implemented process of claim 19 wherein the array 
is is checked before computing the estimated error for any candidate MV to 
determine whether an error value for that candidate MV has already been 
computed, and wherein if the estimated error for that candidate MV has already 
been computed, then it is read back from the array instead of being computed. 

20 21 . The computer-implemented process of claim 1 7 wherein the 

reliability the candidate MVs for the background blocks is determined by 
computing error values for the MVs for each block in the image frame, and 
comparing the computed error values to a first error threshold. 

25 22. The computer-implemented process of claim 17 wherein the 

reliability of the MVs for the second stage is determined by comparing the error 
value of each best candidate MV with a second error threshold, wherein any best 
candidate MV having a computed error value less than the second error 
threshold is deemed to be a reliable MV. 

30 
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23. The computer-implemented process of claim 22 wherein a second 
error threshold is computed as a minimum error value of the spatial and temporal 
neighbor blocks. 

24. The computer-implemented process of claim 1 7 wherein the error 
value of the best candidate MV for the second stage is compared against a third 
threshold value, wherein: 

if the error value is larger than the third threshold value, then the third 
stage of block-based pattern search comprises the entire search range; and 

if the error value is smaller than the third threshold value, then the third 
stage of block-based pattern search comprises a search range consisting of only 
immediate neighbor MVs of the best candidate MV of the second stage. 

25. The computer-implemented process of claim 24 wherein the third 
threshold value is computed as a maximum of the computed error values of the 
spatial and temporal neighbor blocks. 
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